Display apparatus

ABSTRACT

A driving assistance apparatus is provided in which the detection range of a left-front-corner sonar ( 12   a ) located at the vehicle&#39;s left front corner is included in the field of view of a second imaging means ( 14 ) located at the vehicle&#39;s left front corner. When the left-front-corner sonar ( 12   a ) detects a three-dimensional object at the vehicle&#39;s left front corner, an image processing means ( 3 ) synthesizes an image of the image created using a second imaging means ( 14 ) and the images created with four cameras ( 7   a - 7   d ) for imaging the complete periphery of the vehicle, and creates a bird&#39;s-eye-view image ( 40   b ). The detection range of the left-front-corner sonar ( 12   a ) is included within a region of the bird&#39;s-eye image ( 40   b ) on the basis of the image created with the second imaging means ( 14 ).

CROSS REFERENCE TO RELATED APPLICATIONS

This is a continuation of U.S. patent application Ser. No. 16/263,159filed on Jan. 31, 2019, which is a continuation of U.S. patentapplication Ser. No. 14/241,735 filed on Feb. 27, 2014, which is thenational phase of PCT Application No. PCT/JP2012/005321 filed Aug. 24,2012, which claims priority from Japanese Patent Application No.2011-184416 filed Aug. 26, 2011 and Japanese Patent Application No.2011-184419 filed Aug. 26, 2011. The contents of all of theseapplications are incorporated by reference herein in their entireties.

TECHNICAL FIELD

The present invention relates to a drive assistance apparatus that givesnotice of a potential collision with an obstacle during parking.

BACKGROUND ART

Hitherto, there is a known drive assistance apparatus that combinesimages captured using a plurality of cameras into an all-around viewimage indicating the all-around view of a vehicle and displays thecombined image to assist driving (see Patent Literature (hereinafter,referred to as “PTL”) 1, for example).

However, the all-around view image is created by combining the cameraimages projected onto a road surface position, so that athree-dimensional object disappears (dead angle) at a junction boundarybetween adjacent cameras. In general, the junction boundary between thecamera images is set in the vicinity of four corners of a vehicle due torestrictions such as the installation of position of the camera or theangle of view of the camera, or the density of pixels. Such four cornersof the vehicle are also likely to become blind zones of the visuallyobservable area of the driver. For this reason, the deriver may continuedriving without realizing the three-dimensional object in the vicinityof the junction boundary and thus cause a collision between the vehicleand the three-dimensional object.

In order to solve such a problem, in the related art, the position orangle of the junction boundary between camera images is changed inassociation with a sonar unit, a gear, or the like (see PTL 2 and PTL 3,for example).

CITATION LIST Patent Literature PTL 1

International Publication No. WO 00/64175

PTL 2 Japanese Patent Application Laid-Open No. 2007-104373 PTL 3Japanese Patent Application Laid-Open No. 2006-121587 SUMMARY OFINVENTION Technical Problem

However, with the technique of the related art, there is a problem inthat a blind spot still exists in the close proximity of the vehicle (adistance within a few tens of centimeters from the vehicle). Inparticular, there is a concern that a driver may not realize thepresence of the three-dimensional object due to the disappearance of thethree-dimensional object on the bird's-eye view image although thepresence of the three-dimensional object has been detected by the sonarunit, for example.

An object of the present invention is thus to provide a drive assistanceapparatus capable of preventing a three-dimensional object fromdisappearing in the close proximity of a vehicle in a bird's-eye viewimage although the three-dimensional object has been detected.

Solution to Problem

In a drive assistance apparatus according to an aspect of the presentinvention, a sensor includes a detection range that is within the angleof view of a second imaging section, and when the sensor detects athree-dimensional object, an image processing section creates abird's-eye view image by combining an image captured by the secondimaging section and images captured by a first imaging section and setsthe detection range of the sensor to be within a region of thebird's-eye view image based on the image captured by the second imagingsection in the bird's-eye view image.

Advantageous Effects of Invention

According to the drive assistance apparatus of the present invention, itis possible to prevent a three-dimensional object from disappearing inthe close proximity of a vehicle in a bird's-eye view image although thethree-dimensional object has been detected, and thus to bring about theeffect of making it easier for the driver to realize thethree-dimensional object in the close proximity of the vehicle.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a driveassistance apparatus according to Embodiment 1 of the present invention;

FIG. 2 is a diagram illustrating the positions at which first and secondimaging sections are mounted to a vehicle according to Embodiment 1 ofthe present invention;

FIG. 3 is a diagram illustrating the positions at which sonar units aremounted to a vehicle according to Embodiment 1 of the present invention;

FIG. 4 is a diagram illustrating an angle of view of the second imagingsection and a detection range of a sonar unit according to Embodiment 1of the present invention in a horizontal plane;

FIG. 5 is a diagram illustrating a three-dimensional object according toEmbodiment 1 of the present invention;

FIG. 6 is a flowchart illustrating drive assistance processing using thedrive assistance apparatus according to Embodiment 1 of the presentinvention;

FIG. 7 is a diagram illustrating a bird's-eye view image created in stepS65 of FIG. 6;

FIG. 8 is a diagram illustrating a blind spot region generated in thebird's-eye view image of FIG. 7;

FIG. 9 is a diagram illustrating a boundary of the blind spot region ofFIG. 8;

FIG. 10 is a block diagram illustrating a configuration of a driveassistance apparatus according to Embodiment 2 of the present invention;

FIG. 11 is a diagram illustrating the positions at which first andsecond imaging sections are mounted to a vehicle according to Embodiment2 of the present invention;

FIG. 12 is a diagram illustrating the positions at which sonar units aremounted to a vehicle according to Embodiment 2 of the present invention;

FIG. 13 is a diagram illustrating an angle of view of the second imagingsection and a detection range of the sonar unit according to Embodiment2 of the present invention in a horizontal plane; and

FIG. 14 is a flowchart illustrating drive assistance processing usingthe drive assistance apparatus according to Embodiment 2 of the presentinvention.

DESCRIPTION OF EMBODIMENTS Embodiment 1

Hereinafter, a drive assistance apparatus according to Embodiment 1 ofthe present invention will be described with reference to theaccompanying drawings. Meanwhile, in the present embodiment, a vehiclehaving a steering wheel on its right side will be described as anexample. In a case of a vehicle having a steering wheel on its left handside, the left and right are reversed.

FIG. 1 is a block diagram illustrating a configuration of a driveassistance apparatus according to Embodiment 1 of the present invention.

In FIG. 1, drive assistance apparatus 1 includes an imaging electriccontrol unit (ECU) configured to perform image processing and includesvolatile memory 2, image processing section 3, non-volatile memory 4,control section 5, and bus 6 for connecting these components to eachother. Drive assistance apparatus 1 is connected to first imagingsection 7, input section 8, vehicle speed sensor 9, steering sensor 10,gear 11, sonar section 12, display section 13, and second imagingsection 14. Drive assistance apparatus 1 may include input section 8,vehicle speed sensor 9, steering sensor 10, gear 11, sonar section 12,display section 13, and second imaging section 14. Meanwhile, steeringsensor 10 and a steering signal which are illustrated in FIG. 1 may alsobe referred to as “steering angle sensor 10” and a “steering anglesignal,” respectively.

Volatile memory 2 includes a video memory or a random access memory(RAM), for example. Volatile memory 2 is connected to first imagingsection 7. In addition, volatile memory 2 is connected to second imagingsection 14. Volatile memory 2 temporarily stores image data itemsobtained from captured images which are received from first imagingsection 7 and second imaging section 14 at every predetermined time. Theimage data items stored in volatile memory 2 are output to imageprocessing section 3 through bus 6.

Image processing section 3 includes an application specific integratedcircuit (ASIC) or very large scale integration (VLSI) chip, for example.Image processing section 3 is connected to display section 13. Imageprocessing section 3 performs the conversion of a viewpoint on the imagedata items which are received from volatile memory 2 and creates abird's-eye view image in which the image data items received fromnon-volatile memory 4 are superimposed on each other, at everypredetermined time. Image processing section 3 may create a combinedimage in which ordinary images without conversion of a viewpoint arearranged as the bird's-eye view image. The technique disclosed inInternational Publication No. WO 00/64175 can be used as a method ofconverting a viewpoint, for example. Image processing section 3 outputsthe combined images which are created at every predetermined time asdisplay images to display section 13.

Non-volatile memory 4 includes a flash memory or a read only memory(ROM), for example. Non-volatile memory 4 stores various image dataitems such as an image data of a vehicle for which the drive assistanceapparatus is used (hereinafter, referred to as “host vehicle”) and adata table regarding a display method in accordance with drivingsituations. The image data items stored in non-volatile memory 4 areread out in response to a command of control section 5, and is used forvarious image processing using image processing section 3.

Control section 5 includes a central processing unit (CPU) or largescale integration (LSI) chip, for example. Control section 5 isconnected to input section 8, vehicle speed sensor 9, steering sensor10, gear 11, and sonar section 12. Control section 5 controls the imageprocessing of image processing section 3, data read out from volatilememory 2 or non-volatile memory 4, input from first imaging section 7 orsecond imaging section 14, and output to display section 13 on the basisof various signals input from input section 8, vehicle speed sensor 9,steering sensor 10, gear 11, and sonar section 12.

First imaging section 7 includes four cameras. On the other hand, secondimaging section 14 includes one camera. First imaging section 7 andsecond imaging section 14 input images captured at every predeterminedtime to volatile memory 2 of drive assistance apparatus 1. First imagingsection 7 is mounted to a vehicle body so as to be capable of capturingimages of all-around view of a host vehicle. In addition, second imagingsection 14 is installed at a left front corner of the vehicle. Thepositions at which first imaging section 7 and second imaging section 14are mounted to the vehicle body will be described below.

FIG. 2 is a diagram illustrating the positions at which first imagingsection 7 and second imaging section 14 are mounted to a vehicle. Asillustrated in FIG. 2, first imaging section 7 includes front camera 7a, right camera 7 b, left camera 7 c, and rear camera 7 d. For example,front camera 7 a and rear camera 7 d are mounted to the front and rearbumpers of the vehicle body, respectively. For example, right camera 7 band left camera 7 c are mounted to the lower portions of right and leftdoor mirrors of the host vehicle. On the other hand, second imagingsection 14 is mounted to the left front corner of the host vehicle.

Input section 8 includes a touch panel, a remote controller, or aswitch, for example. When input section 8 is formed of a touch panel,the input section may be provided to display section 13.

Vehicle speed sensor 9, steering sensor 10, gear 11, and sonar section12 output a vehicle speed signal indicating the vehicle speed of thehost vehicle, a steering angle signal indicating a steering angle, agear signal indicating the state of a shift lever, a detected signal anddistance signal of a three-dimensional object to control section 5,respectively. Sonar section 12 includes eight sonar units which aremounted to four places of four corners of the vehicle body of the hostvehicle and four places of the front and back of the vehicle body,respectively. The positions at which the sonar units of sonar section 12are mounted to the vehicle body will be described below.

FIG. 3 is a diagram illustrating the positions at which the sonar unitsof sonar section 12 are mounted to a vehicle. As illustrated in FIG. 3,sonar section 12 includes left front corner sonar unit 12 a, right frontcorner sonar unit 12 b, left rear corner sonar unit 12 c, right rearcorner sonar unit 12 d, left front sonar unit 12 e, right front sonarunit 12 f, left rear sonar unit 12 g, and right rear sonar unit 12 h. Asillustrated in FIG. 3, respective horizontal detection ranges 16 e to 16h of left front sonar unit 12 e, right front sonar unit 12 f, left rearsonar unit 12 g, and right rear sonar unit 12 h are set to be narrowerthan respective horizontal detection ranges 16 a to 16 d of left frontcorner sonar unit 12 a, right front corner sonar unit 12 b, left rearcorner sonar unit 12 c, and right rear corner sonar unit 12 d. Next, arelation between the detection range of left front corner sonar unit 12a and an angle of view of second imaging section 14 will be described.

FIG. 4 is a diagram illustrating the angle of view of second imagingsection 14 and the horizontal detection range of left front corner sonarunit 12 a. As illustrated in FIG. 0.4, the angle of view 17 of secondimaging section 14 is set to approximately 180 degrees in a horizontalplane. In addition, detection range 16 a of left front corner sonar unit12 a is included within the angle of view 17 of second imaging section14. That is, the entirety of detection range 16 a of left front cornersonar unit 12 a is included within the angle of view 17 of secondimaging section 14.

It is preferable that second imaging section 14 be mounted furtherupward of the vehicle body than left front corner sonar unit 12 a. Thus,detection range 16 a of left front corner sonar unit 12 a has a tendencyfor being three-dimensionally included within the angle of view 17 ofsecond imaging section 14. In addition, it is preferable that opticalaxes of second imaging section 14 and left front corner sonar unit 12 abe substantially equal to each other. Accordingly, a deviation ofdetection range 16 a of left front corner sonar unit 12 a within angleof view 17 of second imaging section 14 becomes smaller, and thus it ispossible to reduce a concern that detection range 16 a of left frontcorner sonar unit 12 a may partially protrude outside the angle of view17 of second imaging section 14.

Display section 13 includes, for example, a navigation apparatus or adisplay section provided to a rear seat. Display section 13 displays acombined image input from image processing section 3. The combined imagemay be only a bird's-eye view image, or may be an image in which abird's-eye view image and a normal image are arranged in parallel. Whena blind spot is present in the vicinity of a boundary of the bird's-eyeview image, a three-dimensional object disappears. Here, thethree-dimensional object in this embodiment will be illustrated.

FIG. 5 is a diagram illustrating a three-dimensional object in thisembodiment. As illustrated in FIG. 5, Color Cone (registered trademark)having a width of approximately 30 cm, a depth of approximately 30 cm,and a height of approximately 50 cm is assumed to be thethree-dimensional object in this embodiment. When half or more than halfof Color Cone (registered trademark) disappears three-dimensionally on abird's-eye view image, it means that a blind spot is present in thevicinity of a boundary of the bird's-eye view image.

Next, the drive assistance processing using control section 5 will bedescribed.

FIG. 6 is a flowchart illustrating the drive assistance process usingcontrol section 5.

First, as shown in step S61, control section 5 determines whether theshift lever is in a reversed state, on the basis of the gear signalinput from gear 11.

In a case of YES in step S61, image processing section 3 creates abird's-eye view image using an image captured by first imaging section 7and acquired from volatile memory 2, in response to a command of controlsection 5. In addition, as shown in step S62, display section 13displays the created bird's-eye view image in parallel with a rear imageof rear camera 7 d which is acquired from volatile memory 2.

Next, in a case of NO in step S61, as shown in step S63, imageprocessing section 3 creates a bird's-eye view image using an imagecaptured by first imaging section 7 and acquired from volatile memory 2,in response to a command of control section 5, and display section 13displays the created bird's-eye view image in parallel with a frontimage of front camera 7 a which is acquired from volatile memory 2.

Next, as shown in step S64, control section 5 determines whether athree-dimensional object is present at a left front corner of a hostvehicle, on the basis of detected results of sonar section 12. That is,control section 5 determines whether left front corner sonar unit 12 ahas detected a three-dimensional object. In a case of NO in step S64,the processing of step S61 is performed again.

On the other hand, in a case of YES in step S64, as shown in step S65,image processing section 3 newly creates a bird's-eye view image usingthe image captured by first imaging section 7 and an image captured bysecond imaging section 14 and causes display section 13 to display thecreated bird's-eye view image. That is, only when left front cornersonar unit 12 a detects a three-dimensional object, image processingsection 3 creates a bird's-eye view image using images captured by fourcameras 7 a to 7 d of first imaging section 7 and the image captured bysecond imaging section 14. Conversely, when left front corner sonar unit12 a has not detected the three-dimensional object located at the leftfront corner of the host vehicle, image processing section 3 creates thebird's-eye view image so far using only the images captured by fourcameras 7 a to 7 d of first imaging section 7. Differences between thebird's-eye view image created in step S65 and an ordinary bird's-eyeview image created by the images captured by four cameras 7 a to 7 d offirst imaging section 7 will be described later.

Next, as shown in step S66, control section 5 determines whether themoving direction of the host vehicle is the forward direction. At thistime, control section 5 specifies the moving direction of the hostvehicle on the basis of the gear signal input from gear 11. That is,control section 5 determines from the gear signal whether the shiftlever is set to the front.

In a case of YES in step S66, as shown in step S67, control section 5causes display section 13 to display the image captured by secondimaging section 14, instead of the rear image displayed in parallel ondisplay section 13 by step S62 or the front image displayed in parallelon display section 13 by step S63.

After the processing of step S67, or in a case of NO in step S66, asshown in step S68, control section 5 determines whether the terminationof a drive assistance mode has been detected. In a case of YES in stepS68, control section 5 terminates the drive assistance processing. Forexample, when an input of the termination of the drive assistance modeis received from input section 8, control section 5 terminates the driveassistance processing. On the other hand, in a case of NO in step S68,control section 5 performs the processing of step S61 again.

Next, a description will be given of differences between the bird's-eyeview image created in step S65 when left front corner sonar unit 12 adetects the three-dimensional object and an ordinary bird's-eye viewimage created by the images captured by four cameras 7 a to 7 d of firstimaging section 7. FIG. 7 is a diagram illustrating the bird's-eye viewimage created in step S65, using an image.

As illustrated on the left side of FIG. 7, before left front cornersonar unit 12 a detects a three-dimensional object, image processingsection 3 creates bird's-eye view image 40 a using the images capturedby four cameras 7 a to 7 d of first imaging section 7. Image 21 of ahost vehicle is superimposed on the center of bird's-eye view image 40a. Regions 22 to 25 of bird's-eye view image 40 a correspond toviewpoint-converted images of the images captured by front camera 7 a,right camera 7 b, left camera 7 c, and rear camera 7 d, respectively.The junction surfaces of regions 22 to 25 are shown as combinationboundaries 31 to 34, respectively. When left front corner sonar unit 12a detects three-dimensional object 41 in the vicinity of the hostvehicle, a blind spot is generated at combination boundary 34 in thebird's-eye view image itself using the images captured by four cameras 7a to 7 d of first imaging section 7, and thus three-dimensional object41 disappears.

Consequently, when left front corner sonar unit 12 a detects thethree-dimensional object, as illustrated on the right side of FIG. 7,image processing section 3 creates bird's-eye view image 40 b using notonly the images captured by four cameras 7 a to 7 d of first imagingsection 7 but also an image captured by second imaging section 14 thatcaptures an image of the left front corner in which three-dimensionalobject 41 is detected. Regions 22 to 26 of bird's-eye view image 40 bcorrespond to viewpoint-converted images of the images captured by frontcamera 7 a, right camera 7 b, left camera 7 c, rear camera 7 d, andsecond imaging section 14, respectively. Combination boundary 35 betweenfront camera 7 a and second imaging section 14 and combination boundary36 between left camera 7 d and second imaging section 14 are set topositions at which region 26 of the viewpoint-converted image of secondimaging section 14 can include the detection range of left front cornersonar unit 12 a. In other words, combination boundaries 35 and 36 areset outside the detection range of left front corner sonar unit 12 a.Thus, three-dimensional object 41 detected by left front corner sonarunit 12 a does not disappear in the vicinity of combination boundaries35 and 36 in bird's-eye view image 40 b, and the visibility ofthree-dimensional object 41 within region 26 is maintained. Inparticular, since a falling-down direction of three-dimensional object41 in the vicinity of the combination boundary does not rapidlyfluctuate, a driver can view three-dimensional object 41 without feelinga sense of discomfort. In addition, three-dimensional object 41 fallsdown in a direction which is radially away from the vehicle as areference point, and thus it is possible to intuitively ascertain theposition and direction of three-dimensional object 41.

Meanwhile, in order to set three-dimensional object 41 to be distantfrom combination boundaries 35 and 36 as far as possible, it isfundamentally preferable that combination boundaries 35 and 36 be set tobe as close as possible to the angle of view of second imaging section14. On the other hand, if combination boundaries 35 and 36 are set to besubstantially equal to the angle of view of second imaging section 14,glare of the outside of the original angle of view may occur when secondimaging section 14 deviates from its mounting position. For this reason,it is preferable that combination boundaries 35 and 36 be set topositions located approximately several degrees to tens of degreesinward with respect to the angle of view of second imaging section 14.

Next, a blind spot region of the bird's-eye view image created in stepS65 and a blind spot region of an ordinary bird's-eye view image createdby the images captured by four cameras 7 a to 7 d of first imagingsection 7 will be described. FIG. 8 is a diagram illustrating a deadangle region generated in the bird's-eye view image of FIG. 7.

Ordinary bird's-eye view image 50 a created by the images captured byfour cameras 7 a to 7 d of first imaging section 7 is illustrated on theleft side of FIG. 8. Bird's-eye view image 50 b created in step S65 isillustrated on the right side of FIG. 8. In bird's-eye view images 50 aand 50 b, a fixed range from host vehicle image 51 is set to blind spotmeasurement region 52. The fixed range is set to approximately severaltens of cm. For example, the fixed range is set to 50 cm. This indicatesa distance at which the host vehicle can move forward in a creepingmanner at a speed of approximately 3 km/h and stop with sudden braking.

As illustrated on the left side of FIG. 8, in ordinary bird's-eye viewimage 50 a created by the images captured by four cameras 7 a to 7 d offirst imaging section 7, detection range 53 of left front corner sonarunit 12 a partially overlaps blind spot region 54 caused by combinationboundary 34 between front camera 7 a and left camera 7 c. When detectionrange 53 of left front corner sonar unit 12 a overlaps blind spot region54, three-dimensional object 41 disappears within the overlapping range.Consequently, in bird's-eye view image 50 b created in step S65,combination boundaries 35 and 36 are kept away from detection range 53of left front corner sonar unit 12 a using the image captured by secondimaging section 14. Thus, as illustrated on the right side of FIG. 8, inbird's-eye view image 50 b created in step S65, detection range 53 ofleft front corner sonar unit 12 a is separated from blind spot region 55caused by combination boundary 35 between second imaging section 14 andfront camera 7 a and blind spot region 56 caused by combination boundary36 between second imaging section 14 and left camera 7 d so as not tooverlap the blind spot regions. Accordingly, the three-dimensionalobject which is present in the detection range of left front cornersonar unit 12 a is displayed on display section 13 without disappearingon bird's-eye view image 50 b.

Next, a blind spot region caused by a combination boundary of thebird's-eye view image of FIG. 8 will be described. FIG. 9 is a diagramillustrating a boundary of the blind spot region of FIG. 8. In FIG. 9,an example of bird's-eye view image 50 a on the left side of FIG. 8 willbe described.

Boundary line 54 a illustrated on the upper left side of FIG. 9indicates a boundary of blind spot region 54 on the left side of thepaper of FIG. 8. That is, boundary line 54 a indicates an outer edge ofdead angle measurement region 52.

Boundary line 54 b illustrated on the lower left side of FIG. 9indicates a boundary of blind spot region 54 on the lower side of thepaper of FIG. 8. Boundary line 54 b indicates a boundary at which ablind spot is generated when three-dimensional object 41 is presentwithin region 25. When three-dimensional object 41 is moved to the upperside of the paper of FIG. 9 toward combined boundary 34 between frontcamera 7 a and left camera 7 d, half or more than half ofthree-dimensional object 41 disappears (25 cm or more) in a heightdirection. The lowermost position of three-dimensional object 41 in thepaper of FIG. 9 serves as a component of boundary line 54 b. Boundaryline 54 b is indicated by a set of the lowermost positions ofthree-dimensional object 41 in the sheet of FIG. 9 when gradually movingthree-dimensional object 41 to the left side of the sheet of FIG. 9 torepeatedly perform the same processing.

Boundary line 54 c illustrated on the upper right side of FIG. 9indicates a boundary of blind spot region 54 on the right side of thesheet of FIG. 8. That is, boundary line 54 c indicates an outer edge ofhost vehicle image 51.

Boundary line 54 d illustrated on the lower right side of FIG. 9indicates a boundary of blind spot region 54 on the upper side of thesheet of FIG. 8. Boundary line 54 d indicates a boundary at which ablind spot is generated when three-dimensional object 41 is presentwithin region 22. When three-dimensional object 41 is moved to the lowerside of the sheet of FIG. 9 toward combination boundary 34 between frontcamera 7 a and left camera 7 d, half or more than half ofthree-dimensional object 41 disappears (15 cm or more) in a widthdirection. The uppermost position of three-dimensional object 41 in thesheet of FIG. 9 serves as a component of boundary line 54 d. Boundaryline 54 d is indicated by a set of the uppermost positions ofthree-dimensional object 41 in the sheet of FIG. 9 when gradually movingthree-dimensional object 41 to the left side of the sheet of FIG. 9 torepeatedly perform the same processing.

A similar method of determining a boundary is applied to blind spotregions 55 and 56 of bird's-eye view image 50 b on the right side ofFIG. 8, and thus the detailed description thereof will be omitted.

As described above, according to the present invention, the detectionrange of left front corner sonar unit 12 a is included within the angleof view of second imaging section 14. Thus, when left front corner sonarunit 12 a detects a three-dimensional object, image processing section 3combines the image captured by second imaging section 14 and the imagescaptured by four cameras 7 a to 7 d of the first imaging section tocreate bird's-eye view image 40 b and sets the detection range of leftfront corner sonar unit 12 a to be within a range of the bird's-eye viewimage based on the image captured by second imaging section 14 inbird's-eye view image 40 b. That is, image processing section 3 moves ablind spot of the three-dimensional object outside the detection rangeof left front corner sonar unit 12 a. Therefore, although left frontcorner sonar unit 12 a detects a three-dimensional object, it ispossible to prevent the three-dimensional object in the proximity of ahost vehicle from disappearing on a bird's-eye view image of displaysection 13. In particular, in this embodiment, rather than all of thefour corners of the host vehicle are set as positions where thethree-dimensional object in the close proximity of the host vehicle maydisappear on the bird's-eye view image, only the left front side, whichis likely to become a blind spot to a driver in a case of a vehiclehaving a steering wheel on its right hand side. Thus, a selector is nolonger necessary in drive assistance apparatus 1 which can only have alimited number of camera input ports, so that it is possible to preventan increase in size of a control ECU of drive assistance apparatus 1 dueto the selector.

Meanwhile, in the drive assistance processing illustrated in FIG. 6 ofthis embodiment, a bird's-eye view image and an ordinary image (a frontimage or a rear image) are displayed in parallel to provide a pluralityof determination criteria to the driver to thereby improving thevisibility of a three-dimensional object, but it is also possible todisplay the bird's-eye view image alone. That is, at least theprocessing of step S64 and step S65 of FIG. 6 may be performed.Specifically, image processing section 3 combines an image captured byfirst imaging section 7 and an image captured by second imaging section14 into a bird's-eye view image on the basis of common mapping data.Display section 3 displays the bird's-eye view image. According to sucha configuration, the continuity of a display image is maintained beforeand after a combination boundary of the bird's-eye view image, and thusit is possible to prevent a driver from feeling a sense of discomfortand to prevent the generation of a blind spot and the disappearance ofthe three-dimensional object located at the combination boundary of thebird's-eye view image after detecting the three-dimensional object.

In addition, in this embodiment, since a case of a vehicle having asteering wheel on its right hand side is assumed, a three-dimensionalobject in the vicinity of a left front corner, which is likely to becomea blind spot to the driver, is prevented from disappearing on thebird's-eye view image, using second imaging section 14 and left frontcorner sonar unit 12 a which are provided at the left front corner. Onthe other hand, in a case of a vehicle having a steering wheel on itsleft hand side, an object is located on the right front side rather thanthe left front side. That is, in the case of a vehicle having a steeringwheel on its left hand side, an installation position of second imagingsection 14 in this embodiment is replaced by the right front corner, andleft front corner sonar unit 12 a is replaced by front corner sonar unit12 b.

That is, second imaging section 14 captures an image of a front cornerin a direction opposite to the position of the steering wheel of a hostvehicle among four corners of the host vehicle. The detection range ofsonar section 12 detecting a three-dimensional object, which is presentat the front corner in a direction opposite to the position of thesteering wheel of the host vehicle, is included within the angle of viewof second imaging section 14. When sonar section 12 detects thethree-dimensional object, image processing section 3 creates abird's-eye view image by combining the image captured by first imagingsection 7 and the image captured by second imaging section 14 and mayset the detection range of sonar section 12 to be within a region of thebird's-eye view image based on the image captured by second imagingsection 14 in the bird's-eye view image. On the other hand, when sonarsection 12 has not detected the three-dimensional object which ispresent at the front corner in the direction opposite to the position ofthe steering wheel of the host vehicle, image processing section 3creates a bird's-eye view image by combining the images captured byfirst imaging section 7.

Meanwhile, in this embodiment, sonar units which detectthree-dimensional objects located at the front and back of a vehicle insonar section 12 are formed of four sonar units 12 e to 12 f, but atleast two sonar units may be provided in order to detect thethree-dimensional objects located at the front and back of the vehicle.

In addition, in this embodiment, although sonar section 12 is used as athree-dimensional object detecting section for detecting athree-dimensional object, any means such as an infrared sensor may beused as long as it is a sensor that detects a three-dimensional object.

Embodiment 2

Next, a drive assistance apparatus according to Embodiment 2 of thepresent invention will be described with reference to the accompanyingdrawings. A description similar to that in Embodiment 1 will be giventhe same reference numerals and signs, and the detailed descriptionthereof will be omitted.

FIG. 10 is a block diagram illustrating a configuration of the driveassistance apparatus according to Embodiment 2 of the present invention.In FIG. 10, drive assistance apparatus 1 further includes selector 15with respect to the drive assistance apparatus of Embodiment 1. Driveassistance apparatus 1 is connected to second imaging section 14 throughselector 15. Selector 15 may also be included in drive assistanceapparatus 1. Volatile memory 2 is connected to second imaging section 14through selector 15. Control section 5 controls selector 15 based on asignal input from sonar section 12 and selects second imaging section14.

Second imaging section 14 includes four cameras. First imaging section 7and second imaging section 14 input images captured at everypredetermined time to volatile memory 2 of drive assistance apparatus 1.Second imaging section 14 is mounted to each of four corners of avehicle body of a host vehicle. The position at which second imagingsection 14 is mounted to the vehicle body will be described below.

FIG. 11 is a diagram illustrating the positions at which first imagingsection 7 and second imaging section 14 are mounted to a vehicle. Asillustrated in FIG. 11, second imaging section 14 includes left frontcorner camera 14 a, right front corner camera 14 b, left rear cornercamera 14 c, and right rear corner camera 14 d. Selector 15 selects oneof left front corner camera 14 a, right front corner camera 14 b, leftrear corner camera 14 c, and right rear corner camera 14 d on the basisof a command of control section 5. An image captured by the selectedcamera is input to volatile memory 2 of drive assistance apparatus 1 atevery predetermined time.

Sonar section 12 includes four sonar units which are mounted to fourcorners of the vehicle body of the host vehicle, respectively. Thepositions at which the sonar units of sonar section 12 are mounted tothe vehicle body will be described below.

FIG. 12 is a diagram illustrating the positions at which the sonar unitsof sonar section 12 are mounted to a vehicle, respectively. Asillustrated in FIG. 12, sonar section 12 includes left front cornersonar unit 12 a, right front corner sonar unit 12 b, left rear cornersonar unit 12 c, and right rear corner sonar unit 12 d. As illustratedin FIG. 12, respective horizontal detection ranges 16 a to 16 d of leftfront corner sonar unit 12 a, right front corner sonar unit 12 b, leftrear corner sonar unit 12 c, and right rear corner sonar unit 12 d areeach set to equal to or less than 180 degrees.

FIG. 13 is a diagram illustrating an angle of view of left front cornercamera 14 a and a horizontal detection range of left front corner sonarunit 12 a. As illustrated in FIG. 13, a relation between a detectionrange of sonar section 12 and an angle of view of second imaging section14 is similar to the relation shown in FIG. 4 of Embodiment 1. That is,the angle of view 17 a of left front corner camera 14 a is set toapproximately 180 degrees in a horizontal plane. Detection range 16 a ofleft front corner sonar unit 12 a is included within the angle of view17 a of left front corner camera 14 a. That is, the entirety ofdetection range 16 a of left front corner sonar unit 12 a is includedwithin the angle of view 17 a of left front corner camera 14 a.Similarly to Embodiment 1, it is preferable that left front cornercamera 14 a be mounted further upward of the vehicle body than leftfront corner sonar unit 12 a. In addition, it is preferable that opticalaxes of front corner camera 14 a and left front corner sonar unit 12 abe substantially equal to each other.

Meanwhile, in FIG. 13, as an example, the relation between the detectionrange of sonar section 12 and the angle of view of second imagingsection 14 has been described using the angle of view of left frontcorner camera 14 a and the detection range of left front corner sonarunit 12 a, but a similar relation is established with respect to fourcorners of another vehicle. That is, the detection ranges of right frontcorner sonar unit 12 b, left rear corner sonar unit 12 c, and right rearcorner sonar unit 12 d are included within the angles of view of rightfront corner camera 14 b, left rear corner camera 14 c, and right rearcorner camera 14 d, respectively.

Next, the drive assistance processing using control section 5 will bedescribed.

FIG. 14 is a flowchart illustrating the drive assistance processingusing control section 5. The processing of step S71 to step S73 issimilar to that of corresponding step S61 to step S63 of FIG. 6 ofEmbodiment 1. In step S74, control section 5 determines whether athree-dimensional object is present within a predetermined range in thevicinity of a host vehicle, from detected results of sonar section 12.In a case of NO in step S74, the processing of step S71 is performedagain. On the other hand, in a case of YES in step S74, as shown in stepS75, control section 5 specifies a response location of athree-dimensional object which is present within a shortest distancefrom a host vehicle. That is, when the number of sonar units detecting athree-dimensional object is one, among sonar units 12 a to 12 d providedat four corners of the host vehicle, control section 5 determines thatthe corner at which the sonar unit detecting the three-dimensionalobject is disposed is the response location of the three-dimensionalobject. On the other hand, when the number of sonars detecting athree-dimensional object is two or more, among sonars 12 a to 12 dprovided at four corners of the host vehicle, control section 5determines that the corner at which the sonar detecting athree-dimensional object closest to the host vehicle is disposed is theresponse location of the three-dimensional object, on the basis ofdistance signals of the host vehicle and the three-dimensional objectwhich are received from sonar section 12.

Next, as shown in step S76, image processing section 3 newly creates abird's-eye view image using not only an image captured by first imagingsection 7 but also an image obtained by capturing the vicinity of theresponse location of the three-dimensional object closest to the hostvehicle which is specified in step S75 and causes display section 13 todisplay the created bird's-eye view image. That is, image processingsection 3 creates the bird's-eye view image using images captured byfour cameras 7 a to 7 d of first imaging section 7 and an image capturedby a camera of sonar section 12 which corresponds to the responselocation of the three-dimensional object specified in step S75, amongcameras 14 a to 14 d of second imaging section 14. Differences betweenthe bird's-eye view image created in step S76 and an ordinary bird's-eyeview image created by the images captured by four cameras 7 a to 7 d offirst imaging section 7 are similar to the differences described inEmbodiment 1, and thus the detailed description thereof will be omitted.

Next, as shown in step S77, control section 5 determines whether athree-dimensional object is present in the moving direction of a hostvehicle within a shortest distance from the vehicle. At this time,control section 5 specifies the moving direction of the host vehicle onthe basis of a gear signal input from gear 11. That is, control section5 specifies, from a gear signal, that the moving direction of thevehicle is the forward direction when the shift lever is set to thefront and that the moving direction of the vehicle is the backwarddirection when the shift lever is set to the reverse. In addition,control section 5 compares the specified moving direction and theresponse location of the three-dimensional object in step S75 to performthe determination of step S77. That is, when the shift lever is set tothe front and the moving direction of the vehicle is the forwarddirection, control section 5 determines, from the gear signal, whetherthe response location of the three-dimensional object specified in stepS75 is located on the left front side or the right front side. On theother hand, when the shift lever is set to the reverse and the movingdirection of the vehicle is the backward direction, control section 5determines, from the gear signal, whether the three-dimensional objectreaction location specified in step S75 is located on the left rear sideor the right rear side.

In a case of YES in step S77, as shown in step S78, control section 5causes display section 13 to display the image captured by the camerawhich captures an image of the vicinity of the response location of thethree-dimensional object closest to the host vehicle which is specifiedin step S65, in second imaging section 14, instead of the rear imagedisplayed in parallel on display section 13 by step S72 or the frontimage displayed in parallel on display section 13 by step S73. After theprocessing of step S78, or in a case of NO in step S77, the processingof step S79 is performed. The processing of step S79 is similar to thatof step S68 of FIG. 6 of Embodiment 1, and thus the description thereofwill be omitted.

As described above, according to the present invention, the detectionranges of sonar unit 12 a to sonar unit 12 d are included within theangles of view of corner camera 14 a to corner camera 14 d,respectively. When sonar unit 12 a to sonar unit 12 d detect athree-dimensional object, a corner camera corresponding to the sonardetecting a three-dimensional object closest to a vehicle is selected byselector 15. Image processing section 3 creates bird's-eye view image 40b by combining an image captured by the selected corner camera andimages captured by four cameras 7 a to 7 d of first imaging section 7and causes a detection range of the sonar unit to be within a region ofthe bird's-eye view image based on the image captured by the cornercamera in bird's-eye view image 40 b. That is, image processing section3 moves a blind spot of the three-dimensional object outside thedetection range of sonar section 12. Therefore, although sonar section12 detects a three-dimensional object, it is possible to prevent thethree-dimensional object in the close proximity of a host vehicle fromdisappearing on a bird's-eye view image of display section 13.

INDUSTRIAL APPLICABILITY

The drive assistance apparatus of the present invention is useful inthat, when a three-dimensional object located at one of four corners ofa host vehicle is detected using a sonar unit in particular, the drivingassistance apparatus displays the three-dimensional object on abird's-eye view image without disappearance of the three-dimensionalobject, which in turn, allows the driver to easily realize thethree-dimensional object.

REFERENCE SIGNS LIST

-   1 Drive assistance apparatus-   3 Image processing section-   5 Control section-   7 First imaging section-   12 Sonar section-   14 Second imaging section

1: A control method for a control device configured to be mounted in avehicle, the control device being configured to be connected to a firstcamera, a second camera, a third camera, a sensor, and a display, thefirst camera being configured to capture first images of first ambientview at a first out portion of the vehicle, the second camera beingconfigured to capture second images of second ambient view at a secondout portion of the vehicle, the third camera being configured to capturethird images of third ambient view at a third out portion of thevehicle, a first distance between the first out portion and the thirdout portion being longer than a second distance between the first outportion and the second out portion, the first distance being longer thana third distance between the second out portion and the third outportion, the sensor having a detection area at the second out portion ofthe vehicle, and configured to detect a three-dimensional object withinthe detection area, and the display being configured to display at leasta first one image and a second one image on a screen, the control methodcomprising: displaying the first one image having at least a firstregion and a second region on the screen of the display, the firstregion corresponding to one of the first images, the second regioncorresponding to one of the third images, when the sensor does notdetect a three-dimensional object at the second out portion of thevehicle; and displaying the second one image having at least a thirdregion, a fourth region, and a fifth region on the screen of thedisplay, the third region corresponding to one of the first images, thefourth region corresponding to one of the second images, the fifthregion corresponding to one of the third images, the fourth region ofthe second one image including at least a part of the detection area ofthe sensor, the fourth region of the second one image including at leasta part of the three-dimensional object, when the sensor detects athree-dimensional object at the second out portion of the vehicle. 2:The control method according to claim 1, wherein the sensor and thesecond camera respectively include optical axes that substantially matcheach other. 3: The control method according to claim 1, wherein thefirst one image includes a first bird's-eye view image that combines oneof the first images and one of the third images. 4: The control methodaccording to claim 3, wherein the first one image includes an image ofthe vehicle on a center of the first bird's-eye view image. 5: Thecontrol method according to claim 4, wherein the first one image doesnot include one of the second images in the first bird's-eye view image.6: The control method according to claim 1, wherein the second one imageincludes a second bird's-eye view image that combines one of the firstimages and one of the third images. 7: The control method according toclaim 6, wherein the second one image includes an image of the vehicleon a center of the second bird's-eye view image. 8: The control methodaccording to claim 1, wherein the control device comprises a processor.9: The control method according to claim 1, wherein an angle of view ofthe second camera is set to approximately 180 degrees in a horizontalplane. 10: The control method according to claim 1, wherein thedetection area of the sensor is set to be equal to or less than 180degrees. 11: A control device configured to be mounted in a vehicle, thecontrol device comprising: an input configured to be connected to afirst camera, a second camera, a third camera, and a sensor, the firstcamera being configured to capture first images of first ambient view ata first out portion of the vehicle, the second camera being configuredto capture second images of second ambient view at a second out portionof the vehicle, the third camera being configured to capture thirdimages of third ambient view at a third out portion of the vehicle, afirst distance between the first out portion and the third out portionbeing longer than a second distance between the first out portion andthe second out portion, the first distance being longer than a thirddistance between the second out portion and the third out portion, thesensor having a detection area at the second out portion of the vehicle,and being configured to detect a three-dimensional object within thedetection area; an output configured to be connected with a display, thedisplay being configured to display at least a first one image and asecond one image on a screen, wherein: when the sensor does not detect athree-dimensional object at the second out portion of the vehicle, thecontrol device displays the first one image having at least a firstregion and a second region on the screen of the display, the firstregion corresponding to one of the first images, the second regioncorresponding to one of the third images; and when the sensor detects athree-dimensional object at the second out portion of the vehicle, thecontrol device displays the second one image having at least a thirdregion, a fourth region, and a fifth region on the screen of thedisplay, the third region corresponding to one of the first images, thefourth region corresponding to one of the second images, the fifthregion corresponding to one of the third images, the fourth region ofthe second one image including at least a part of the detection area ofthe sensor, the fourth region of the second one image including at leasta part of the three-dimensional object. 12: The control device accordingto claim 11, wherein the sensor and the second camera respectivelyinclude optical axes that substantially match each other. 13: Thecontrol device according to claim 11, wherein the first one imageincludes a first bird's-eye view image that combines one of the firstimages and one of the third images. 14: The control device according toclaim 13, wherein the first one image includes an image of the vehicleon a center of the first bird's-eye view image. 15: The control deviceaccording to claim 14, wherein the first one image does not include oneof the second images in the first bird's-eye view image. 16: The controldevice according to claim 11, wherein the second one image includes asecond bird's-eye view image that combines one of the first images andone of the third images. 17: The control device according to claim 16,wherein the second one image includes an image of the vehicle on acenter of the second bird's-eye view image. 18: The control deviceaccording to claim 11, further comprising a processor. 19: The controldevice according to claim 11, wherein an angle of view of the secondcamera is set to approximately 180 degrees in a horizontal plane. 20:The control device according to claim 11, wherein the detection area ofthe sensor is set to be equal to or less than 180 degrees.